This invention relates generally to testing performance of computer systems, and more particularly, to methods and systems for testing performance of biometric authentication computer (BAC) systems.
During authentication transactions, BAC systems execute 1:1 verification matching algorithms stored therein to confirm whether a person is who he or she claims to be or, alternatively, execute 1:N matching algorithms stored therein to identify an unknown person or determine if a person is already in a database. These matching algorithms typically compare a series of probes against a gallery of data records stored in the BAC system. The identity of a person is verified as the result of a successful 1:1 verification, or determined as the result of a 1:N identification when biometric data included in a probe matches biometric data included in a data record corresponding to the person. Data generated as a result of these comparisons may be used to determine BAC system performance metrics such as, but not limited to, accuracy and false accept rate.
Biometric data test records each include a pair of biometric records. A first record of the pair is known as a probe and is used in test matching transactions. A second record of the pair is known as a seed and is inserted, or otherwise ensured to be in the gallery when the probe is processed. While testing BAC system performance, the gallery is searched for the seed records using the known matching probes such that the rate at which seed records are not detected as well as the frequency that spurious matches are detected can be determined. However, because accuracy and selectivity of matching algorithms degrade during 1:N transactions as the number of data records in the gallery increases, known testing techniques cannot be effectively implemented in BAC systems responsible for managing hundreds of millions of identities.
It has been known to manually generate test data for testing BAC system performance by obtaining probe-seed record pairs of biometric samples from test volunteers and enrolling the seed record of the pair into an existing gallery of biometric data records. However, manually generating such probe-seed record pairs as test data is known to be very expensive. Moreover, because such test data is generally valid for only a single blind test and most BAC systems require regular testing to validate performance, regular testing of the performance of BAC systems is generally considered to be prohibitively expensive. Furthermore, although manually generated test data provides some measurement of BAC system performance, the results may not be representative of performance with operational data and may be of little value.